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Abstract. We use the negative replica method, which was originally devel- 
oped for the study of overfrustation in disordered system, to investigate the 
statistical behaviour of the cost function of minimax games. These games 
are treated as hierarchical statistical mechanical systems, in which one of the 
components is at negative temperature. 



(PACS 61.43FS, 64.60Cn, 89.90+h) 



1. Introduction 

The theory of spin glasses has found interesting appHcations in several branches 
of science [Q . In the theory of combinatorial optimization it inspired the invention 
of the so called simulated annealing heuristic search technique With the help 
of the replica method, several authors ^, || managed to obtain analytical insight 
about optimal solutions of some hard optimization problems. The interest in these 
studies was driven by the fact that many of these problems were members of the NP 
complexity class, which means that to check their solutions requires only polynomial 
time, but to find them is presumably much harder. 

NP is among the first few members of a hierarchy of complexity classes of in- 
creasing difhculty, the polynomial-time hierarchy PH of Meyer and Stockmeyer 
|6|. Examples of problems from this hierarchy are adversary games, where the first 
player tries to minimize the objective function while the second one tries to maxi- 
mize it. In this paper we treat the case in which one of the players has control over 
the spins of a spin glass, while the other controls the external magnetic field, and 
the objective function is the energy of the spin configuration. 

The standard machinery of statistical mechanics provides information on the 
ground state (minimum of energy), as the temperature approaches zero. To study 
the maximum, we need to approach zero from negative direction. Fortunately, this 
step can be incorporated into the replica method by allowing the number of replicas 
to be negative. The method of negative replicas was invented by Dotsenko, Franz 
and Mezard to study partial annealing and overfrustation in disordered systems [Q . 
(Some related works are [|[ ||, 0.) We use this framework for the investigation 
of minimax games. 

In Sec. II we give a short, nontechnical description of the polynomial-time hi- 
erarchy of complexity classes. In Sec. Ill we apply this extension of the replica 
method for three simple models. Sec. IV contains an extension of the negative 
replica method for multi-move games. 
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2. The polynomial-time hierarchy 

In this section we closely follow the exposition of Stocknicyer . To formulate a 
rough definition of the complexity classes it is easier to use decision problems than 
optimization ones. We define the complexity class P as those problems which are 
solvable by a deterministic (and sequential) computer in time bounded by some 
polynomial of the size of the problem. Of course, one should spell out in a little 
more detail the kind of computers used (usually a Turing or Random Access Ma- 
chine), however, the class P is remarkably stable with respect to changes of the 
computational model. 

The definition of the class NP is similar, but in this case the use of nondeter- 
ministic computers is allowed. The nondeterministic model of computation is more 
powerful than the deterministic one. Let us take for example the most representa- 
tive problem of NP, the satisfiability of an arbitrary Boolean expression. If there 
is an assignment of truth values to the variables of the expression such that the 
expression evaluates to 'true', then a nondeterministic computer is able to verify 
that in polynomial time. In the first few steps it correctly guesses that assignment, 
and then by a deterministic algorithm it verifies that the assignment indeed satisfies 
the Boolean expression. These steps take only polynomial time. So NP contains 
those problems, whose solutions, if exist, can be checked in polynomial time. A 
basic conjecture of computer science is that the inclusion P C NP is proper, i.e. 
there are problems easy to check but hard to solve. 

In the case of spin glasses the decision problem is that given a Jij coupling 
constants matrix and a number K, is there any spin configuration Si such that 
Ej{si) = J2ij JijSiSj < K. (To keep the size of the problem under control, Jij 
should take only discrete (maybe ±1) values). A closely related problem class is 
co-NP, the complement of NP. Here the task is to recognize those problems which 
has no solution. For example in the spin glass case one needs to prove that there 
is no spin configuration with energy less than a given constant. It is unlikely that 
such proof of polynomial length exists for a random matrix , so it is believed 
that NP ^ co-NP. 

Optimization problems requires the ability to solve both NP and co-NP prob- 
lems. To prove that is the minima of Ej(s), one should find first s such that 
Uq = Ej{s), then solve a co-NP problem proving that there is no such s that 
Ej{s) < Uo. 

Several ways exist to obtain problems harder than NP. The most obvious is 
to allow more (say exponential) time for the computation. A more subtle way 
to increase the power of the computational model is the use of oracle machines. 
They have an additional instruction 'Call-Oracle' . When the machine executes 
this instruction, it presents the oracle a problem from the oracle's problem class 
for which the oracle gives returns the solution or gives 'no' answer in a single step. 
The power of an oracle computer depends on the oracle's problem class C. Since 
the oracle recognizes non-membership in C, too, the oracles C and co-C have the 
same computational power. In this manner, NP(C) (resp. P(C)) is defined as the 
decision problem class, which satisfiability can be decided by a nondeterministic 
(resp. deterministic) computer with oracle C in polynomial time. 

By denoting P= E^, the polynomial-time hierarchy is defined as 



Er = NP(Ef_i), Af = P(Sf_i), nf^co-Sf. 
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Members of this hierarchy occurs in problems involving the alternation of existential 
and universal quantifiers. The satisfiability of the Boolean formula /(x) (i.e. / € 
NP) means 3x/(x), while its non-satisfiability (i.e. / € co-NP) is the same as 
Vx-i/(x). Boolean formulas 3xiVx2...3x2i+i/(xi, X2, ...) or 

3xiVx2...Vx2i^/(xi, X2, ...) with (fc— l)-fold alternation of existential and universal 
quantifiers provides natural examples for problems from Sj^. The determination of 
the satisfiability of such formulas can be described as a game between two adversary 
players. The first player's objective is to satisfy the formula, while the second one 
tries to set the variables X2, X4,.., so that the formula is not satisfied. 

An optimization problem from the polynomial-time hierarchy is the determina- 
tion of the outcome 

M = maxmin ...c(xi, X2...) 

of a minimax game. For many functions c(xi,X2...), the computation of M is a 

^fe+i type problem if there are fc — 1 alternation of the the mm and max operators. 
In the next section we treat the case where Xi and X2 represent sets of discrete spin 
variables and c(xi,X2) is the energy function of spin configurations. 



3. Spin games 



In this section we study two-move minimax games. The objective function is 
denoted by H{u, v), where u and v are two sets of variables. The first player (the 
minimizer) controls the u variables, while the second one (the maximizer) controls 
the V variables. If both players play optimally, then the outcome of the game is 



M = inf ( sup H{u, v) 



To apply the methods of statistical mechanics, inf„ h{u) (resp. sup^ H{u, v)) is 
replaced by the free energy of a system with Hamiltonian h (resp. H) at low 
positive (resp. negative) temperature. For that purpose we introduce 



--^ln^exp-/3„ I -^ln^exp/3„i?('u, 

{«} \ {v} 



V) 



(1) 



^" w Vw 



= --^ lim - 



J2 exp/3„ ^(""'«")| -1 

,{u^,v°'} {a, a} 



There are n repHcas of u and nk = —nfHu/Pv copies of v. If /3u,Pv 00, then 
M{(3u,Pv) M. (At least if the zero temperature entropy vanishes, which is true 
even in the mean field theory of spin glasses.) 

To gain some experience with the method of negative replicas, we apply it first 
for the non-random Hamiltonian 



(2) 



I..N. 
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In this case the appUcation of the n — > Umit is not necessary, so the u spins 
are not repUcated. The partition function is 



(3) Z=J2^^PPvJ2Hiu,v") = 

a. 

iV^d^ exp I -N [^xy - log (2 cosh [PM9 + ^/k)]) ' 

-fclog (2cosh[/3„(/i + y)]); 



/ 



The large /3 saddle point equations arc 

yo 

2 



tanh 



I3u {9+^)] ~ sign (-5 + 



(4) 

^ = tanh [(3y {h + yo)] w sign {h + yo) . 
Using log (2cosh/32;) « /3|^;| for /? 1, one can check that 
(5) hm log Z = min [2u sign (2u + /i) + + sign {2u + h)] , 

/3„,/3„^cx> Npu ue[-l,l] 

where the last expression is the outcome of the game if both players play optimally, 
since at optimal play v = sign (2u + h). 




g=-3.0 
-m— g=-1.5 
g=0.0 
g=1.5 
g=3.0 



Expected outcome in constant magnetic field 



In the next example random magnetic field has been added to the model: 

(6) H{u,v) = ^ (J:v^^+J:{g + gi)ui + J2{h + hi)vi, 
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where hi — gi = Q and ~ gf ^ I. After some tedious but standard calculations, 
we obtain that M(/3„, f3y) is equal to the saddle-point value of 



(7) 
(8) 



1 



-pq- 



dz 



'Pv 



^2tt 
dw 



e 2 log (^2 cosh -/3„(VJj 



Jp + g + z] 



log (2 cosh l3v{Vjq + h + w) 



with respect to p and q. The saddle point equations are 



(9) 
(10) 



po = 2Vj 
qo = 2\/j 



dw 



^2tt 
dz 



ri"'^ tanh 



^ tanh 



l3y{Vjqo + h + w) 
- Pui^Jpo +9 + z) 



The numerical solution of these equations is presented on the following graph: 




<— g=-3.0 
g=-1.5 
^ g=0.0 
^ g=1.5 
^ g=3.0 



-2 2 

h 

Expected outcome in random magnetic field 



Since the Hamiltonian is fairly simple, the expression can be derived 
without the use of rephcas, too. For that purpose, we assume that M(/3„,/3t,) re- 
ceives its dominant contribution from spin configurations where the u spins' average 
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magnetisation is u. Then 



-1 



ndg. 



/27r 



log' 



{«=} 



exp 



where fv{u) is the free-energy of the Vi spins in the external field of the Ui spin 
variables: 

dtii 1 2 

ze-i'" log (2cosh [/3^(2Jm + /i + w)]). 



(11) 



h{u) = - 

Pv 



M{fiu,Pv) evaluates to 
-1 



(12) 



dz 



e 5^ log (2cosh [-/?„(g + 2:) + iA]) 



dill 1 2 

ze-s'" log (2 cosh [/3„(2Ju + /i 



/27r 



^)]) 



This formula should be computed at its saddle-point value with respect to A and its 
minimum with respect to u € [—1, 1]. After the change of variables u = q/{2^/j) 
and iX — j3u^/Jp, the expressions and (|l2|) coincide. Since we managed to 
evaluate M{j3ml3v) without the use of replicas, too, this example is certainly not 
the most impressive application of the negative replica method. Nevertheless, this 
model provides an example where one can analytically prove that the replica method 
works. 

Finally, we attempt to treat the case of a spin-glass type objective function 
(13) Hj{si,hi)^ ^ J^jSiSj+g ^ h^s^, 

i<i<j<N l<i<N 

where J,i is random variable with Gaussian distribution 



±1, 



±1, 



(14) d^i{J,j) = y/N/2Jexp {-J^ /2N)dJ,j . 

The minimizer makes the first move and controls the hi variables, while the maxi- 
mizer makes the second move and controls the Si spins. The partition function of 
this system is 



i<k 



J n (\/^dQao6/3)exp-7v|-nfc^ + ^ J2 Q 



2 

aahjS 



aa<b() 



-log expPv pv Y Q-'-bps^"^'^' -sY^'^Y.^^" 

{S»°,_ff»} '- aa<bl3 a a 

where k = —Pu/Pv In the replica symmetric approximation 

Qaaap = P, Qaabp = 9 for a ^ b, 
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Zn k equals to 



Zn,k — 




/ n(i 




' aa<bp 


-log J 


f dx _ 





nk n{n — 2 nk{k — 1) 



-P 



dy 

, t 

\/2^ 



X I [2 cosh {p^i^x + ^ff~^y + g))] ^ + [2 cosh {I3„{^x + ^ff~^y - g))] '-'j^^ 



From this equation one obtains the expected outcome of the game: 



= ^(l + fcg2 + (l_fc)p2_2p) + 



iZ„M - 1) 



1 



dx 



[2 cosh {I3„{y^x + Vp-QV -9))]^ 



where the last expression should be evaluated at its saddle point. This expression 
is very similar to the free energy of a spin glass at the one stage replica symmetry 
breaking approximation [p^ . Indeed, Qaabp might be regarded as a nfc x nk matrix 
broken into blocks of size kx k. However, here fc is a fixed negative number. m{p, q) 
has a minimum at p = g = 1 on the line p — q. In this approximation 



(15) 



m 



mimraax 



id) 



dx 
V2^ 



A better approximation is achived if we search for the saddle point on the {p, q) 
plane. Since the first term of m scales as 0(/3) as /3 ^ 00, while the second has 
finit limes, the saddle point should be on the curve 1 + kq^ + (1 — k)p^ — 2p. We 
evaluated numerically m as the function of g. We plot the function mminimax{g) 
(solid line on the figure). 
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Expected outcome of a spin-glass type minimax game 



The exact value oimminimaxi.9) is smaller than mspingiassio), (where m,spingiass{g) 
is the maximal value of ^JijSiSj + gJ2^i)' since the minimizer trios to set hi 
into directions least favorable for the maximizer, while the constant magnetic field 
is equivalent to a randomly choosen hi configuration. However, m,ninimax{g) > 
(rrispinglass (0)), since one of ±J2 hiSi is always nonnegative. 

i9)>i9- 

mspingiass{Oj) also holds, since if the spins are set to the same direction as hi, then 
the contribution of ^ JijSiSj cannot be less than —mspingiass{0) by the symmetry 
of the couplings Jij . We also expect that niminimax (g) converges tog- nispingiass (0) 
as 5 — > DO. These considerations provide upper and lower bounds for mminimaxig) 
(dotted lines on the figure). Unfortunately, the lower bound is violated for small g, 
while its assimptotics is correctly reproduced. It would be interesting to know if a 
better, replica symmetry breaking solution would cure this problem. 

4. Multi-move games 

Up to this point only two-moves games were treated. The extension for multi- 
move games is straightforward. For example, the outcome of the four-move game 



M = inf <^ sup 



inf I sup H{u, V, w, z) 



IS 



lim — — lim 

/3u,u,ra,z— >oo p„ n— »0 



J2 cxp/3^ ^ /f(u",t;"«,w««^,0"«'5T) 

where the ranges of the indices are \a\ = n, \a\ = —(3u/Pv, \P\ = —Pv/Pv 

-Pw/Pz- 
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The limit Pu,v,w,z oo corresponds to the optimal strategies of the players. 
Finite (3 simulates non-exact optimization, i.e. players with bounded computational 
capabilities. An interesting case is when one player's temperature is infinite, so 
the other ones play against random moves. Such games are called 'games against 
Nature' [||. 

5. Discussion 

In the previous sections we used the method of negative replicas to examine opti- 
mization problems arising in minimax games. Such games provide examples of very 
difficult combinatorial problems. In principle our method is able to estimate the 
expected outcome of some adversary games. Unfortunately, due to the complexity 
of the calculations emerging in problems of spin glass type, we manage to treat 
only fairly simple optimization problems. Nevertheless, the method of negative 
replicas provides a natural framework to treat game theoretical problems with the 
machinery of statistical mechanics. 
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